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Embedding Data in Material 

Background of the Invention 

Field of the Invention 

The present invention relates to embedding data in material 
5 "Material" as used herein means information material represented by 

information signals which includes at least one or more of image material, audio 
material. Image material is generic to still and moving images. 
Description of the Prior Art 
Steganographv 

1 0 Steganography is the embedding of data into material such as video material, 

audio materia] and data material in such a way that the data is imperceptible in the 
material. 

Data may be embedded as a watermark in material such as video material, 
audio material and data material. A watermark may be imperceptible or perceptible in 
15 the material. 

A watermark may be used for various purposes. It is known to use watermarks 
for the purpose of protecting the material against, or trace, infringement of the 
intellectual property rights of the owner(s) of the material. For example a watermark 
may identify the owner of the material. 

20 Watermarks may be "robust" in that they are difficult to remove from the 

material. Robust watermarks are useful to trace the provenance of material which is 
processed in some way either in an attempt to remove the mark or to effect legitimate 
processing such as video editing or compression for storage and/or transmission. 
Watermarks may be "fragile" in that they are easily damaged by processing which is 

25 useful to detect attempts to remove the mark or process the material. 

Visible watermarks are useful to allow e.g. a customer to view an image e,g. 
over the Internet to determine whether they wish to buy it but without allowing the 
customer access to the unmarked image they would buy. The watermark degrades the 
image and the mark is preferably not removable by the customer. Visible watermarks 
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are also used to determine the provenance of the material into which they are 
embedded. 

It is known to embed data in material. It is desirable to do that and allow the 
data to be removed from the material to avoid degrading the material. It is desirable to 
minimise any charges to the material needed to embed the data in it to avoid degrading 
the material. It is known to combine the data with the material, the data being scaled 
by a scaling factor which is chosen according to desired properties of the data when 
combined with the material. Those properties include one or more of: concealing the 
data in the material; making the data perceptible in the material: making the data, when 
embedded in the material, resistant to processing which, intentionally and 
unintentionally, removes or damages the embedded data. 

SUMMARY OF THE INVENTION 

According to a first aspect of the invention, there is provided a method of 
embedding data in material, the method comprising 

combining a representation of the material with a function of the data and a 
scaling factor; wherein 

the scaling factor is generated as a function of a trial decoding of the material, 

the trial decoding comprising processing the material to recover data therefrom. 

Thus the scaling factor can be chosen on the basis of an estimate of the result 
of a process (e.g. decoding) which will be performed on the combined material and 
data in practice so as to increase the likelihood that the data is recoverable from the 
material. 

An embodiment of the first aspect of the invention further comprises the steps 
of: 

combining, as a trial, a representation of the material with a function of the data and a 
trial scaling factor; and 

performing, as a trial, a predetermined process on the combined material and data; 

wherein the scaling factor is generated as a function of a trial decoding of the 
processed combined material and data. 

Thus the scaling factor can be chosen on the basis of an estimate of the result 
of a process (e.g. JPEG processing) which could be performed on the combined 
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material and data in practice and which may damage the embedded data, so as to 
increase the likelihood that the data will not be damaged by such processing . 

According to the first aspect of the invention, there is also provided a method 
of embedding data in material the method comprising the steps of: 

producing transform coefficients Ci representing a transform of the material; 

producing a pseudo random symbol sequence (PRSS) having L symbols Pi of 
values +1 and -1; 

calculating the correlation S=E Ci.Pi, for i=l to i=L; and 

calculating modified coefficient values Ci' = Ci + a * Pi, where a is calculated 
dependent on S and the value of the data bit to be embedded in the coefficient. 

Preferably 

ct= (<x ! + offset) 

where a' + offset is a function of the data bit to be embedded in the coefficient, 
and the method comprises the step of calculating modified coefficient values 
Ci* = Ci + (ct'+ offset)*Pi where 

a' =0 if S is positive and the data to be concealed is a bit of a first value, 

a' =0 if S is negative and the data to be concealed is a bit of a second value, 

and otherwise a' is a function of S such that 2 Ci'.Pi has the correct sign to 
represent the bit to be encoded. 

It will be noted that the calculation of the correlation S=S Ci.Pi, for i=l to i=L 
is a form of trial decoding as in the said first aspect and the scaling factor is chosen in 
dependence on that correlation. 

A further aspect of the invention provides a computer program product 
arranged to carry out one of the aforesaid methods when run on a computer. 

The invention also provides corresponding apparatus in other aspects of the 
invention. 

According to a second aspect of the present invention, there is provided a 
method of embedding data in material, comprising the steps of: 
producing transform coefficients Ci of the material: 

comparing the magnitudes of the coefficients with a threshold value T; and 
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producing, from the coefficients Ci and the said data modified, coefficient 
values Ci' which are modified by respective information symbols of a pseudo random 
symbol sequence modulated by the said data to be embedded; 

wherein the said step of producing modified coefficient values does not use 
5 coefficients of magnitude greater than the said threshold T and does not use the 
corresponding information symbols. 

The data is detected at a decoder by correlating a pseudo random symbol 
sequence with the material in which the data is embedded. The data is represented by 
the sign of the correlation function. By not using, during embedding, coefficients 
10 which have a value greater than the threshold, any changes necessary to alter the 
coefficients to achieve the appropriate sign of the correlation value to represent a bit 
of the concealed data are minimised. 

According to the second aspect of the present invention, there is also provided 
a method for detecting data embedded in material, the detecting method comprising 
1 5 receiving transform coefficients of the material; 

comparing the magnitudes of the received coefficients with a threshold value 

T: and 

correlating, the said coefficients with a respective symbols of a pseudo random 
symbol sequence to detect the said data, wherein the correlating step does not use 
20 coefficients of magnitude greater than the said threshold T and corresponding symbols 
of the pseudo random symbol sequence. 

Thus the detecting method is complementary to the embedding method. 
The second aspect of the invention also provides the following a), and b): 
a) Apparatus for embedding data in material comprising a transformer for 
25 producing transform coefficients Ci of the material: 

a comparator for comparing the magnitudes of the coefficients with a threshold 
value T; and 

a combiner for producing, from the coefficients Ci and the said data, modified 
coefficient values Cr which are modified by respective information symbols of a 
30 pseudo random symbol sequence modulated by the said data to be embedded, wherein 
the combiner does not use coefficients of magnitude greater than the said threshold T 
and does not use the corresponding information symbols; 
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b) Apparatus for detecting data embedded in material comprising an input 
for receiving transform coefficients of the material; 

a comparator for comparing the magnitudes of the received coefficients with a 
threshold T; and 

a correlator for correlating, the said coefficients with respective symbols of a 
pseudo random symbol sequence to detect the said data, wherein the correlation does 
not use coefficients of magnitude greater than the said threshold T and the 
corresponding symbols of the pseudo random symbol sequence. 

Yet further according to the second aspect of the invention, there is provided a 
method of detecting data embedded in material the method comprising; 

receiving transform coefficients of the material: 

comparing the magnitudes of the received coefficients with a threshold Tclip; 

clipping, to a magnitude Tclip, the magnitude of coefficients of magnitude 
greater than the said threshold Tclip; and 

correlating the clipped and undipped coefficients with a pseudo random 
symbol sequence to detect data embedded in the material. 

Yet further, apparatus according of the second aspect for detecting data 
embedded in material, comprises: 

an input for receiving transform coefficients Ci' of the material; 

a comparator for comparing the magnitudes of the received coefficients with a 
threshold Tclip; 

means for clipping, to a magnitude Tclip, the magnitude of coefficients of 
magnitude greater than the said threshold Tclip; and 

a correlator for correlating the clipped and undipped coefficients with a 
pseudo random symbol sequence to detect data embedded in the material. 

This further aspect of the invention may involve only the detecting method and 
operates independently of the embedding method. By clipping large value coefficients 
to a preset smaller value, such coefficients no longer dominate the value of the 
correlation function needed to decode the embedded data. 

However, preferably, there is provided: 

a) A method of embedding data in material, the method comprising 
receiving transform coefficients Ci representing the material; 
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comparing the magnitudes of the said transform coefficients Ci with a 
threshold Tclip; 

clipping, to the magnitude Tclip. the magnitudes of those of the coefficients 
having a magnitude exceeding Tclip to produce clipped coefficients; and 

producing modified coefficients Ci' of values dependent on a scaling factor and 
the data to be embedded, and the scaling factor is calculated using the said clipped 
coefficients and the coefficients Ci of magnitude less than Tclip. 

b) Apparatus for embedding data in material the apparatus comprising: 

an input for receiving transform coefficients Ci representing the material; 

a comparator for comparing the magnitudes of the said transform coefficients 
with a threshold Tclip; 

a clipper for clipping, to the magnitude Tclip, the magnitudes of those of the 
coefficients having a magnitude exceeding Tclip: and 

a processor for producing modified coefficients Ci' of values dependent on a 
scaling factor and the data to be embedded, and the scaling factor is calculated using 
the said clipped coefficients and the coefficients Ci of magnitude less than Tclip. 

Thus by clipping large value coefficients to a smaller value such coefficients 
no longer dominate the value of the function used to calculate the scaling factor. 

The invention also provides a computer program product arranged to carry out 
one of the aforesaid methods when run on a computer. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The above and other objects, features and advantages of the invention will be 

apparent from the following detailed description of illustrative embodiments which is to 

be read in connection with the accompanying drawings, in which: 

Figure 1 is a schematic block diagram of an embodiment of a watermarking 
system according to the present invention: 

Figure 2 is a schematic block diagram of another embodiment of a 
watermarking system according to the present invention: 

Figure 3A is a schematic diagram of a wavelet transform showing the 
relationship of the symbols of a pseudo random symbol sequence to coefficients ; 

Figure 3B is a flow diagram of calculations performed by the system of Figure 
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Figure 4 is a schematic block diagram of an illustrative watermark decoding 
and removal system: 

Figure 5 is a flow diagram of calculations performed by the watermark 
remover and decoder of Figure 4; 

Figure 6 is a schematic block diagram of a further embodiment of a 
watermarking system according to the present invention: 

Figures 7 to 10A are schematic block diagram of subsystems of the system of 
Figure 6; 

Figures 10B and 10C are flow diagrams illustrating a process for calculating a: 
Figure 11A is a flow diagram of a modification, in accordance with the 
invention, of the flow diagram of Figure 3B; 

Figure 1 IB is a diagram showing the relationship of coefficients Q and 
symbols P, of a pseudo random symbol sequence: 

Figure 12 is a flow diagram of a modification, in accordance with the 
1 5 invention, of the flow diagram of Figure 5: 

Figure 13 is a flow diagram of another modification, in accordance with the 
invention, of the flow diagram of Figure 5; 

Figures 14 and 15 are diagrams explaining wavelet transforms: and 
Figures 1 6 and 17 are diagrams showing a UMID and a data reduced UMID. 
20 Description of the Preferred Embodiments 

Overview 

Referring to Figure 1, in the shown illustrative watermarking system, a spatial 
domain image I produced by a source 1 is combined with watermark data Ri to 
produce a spatial domain watermarked image I\ The watermarked image is produced 
25 by an embedder 3 according to the equation 

Ci'=Ci + a. Ri 

where Ci and Ci' are, for example, wavelet transform coefficients of the image, 
and a is a parameter which is also referred to herein as a scaling factor, a is chosen in 
this example so that the watermark is imperceptible in the image and to resist removal 
30 of the watermark by unauthorised processing. It is thus desirable that a has the 
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smallest value which achieves that. If a is too big the watermark is perceptible in the 
image; if it is too small the mark may not survive processing of the image. 

In accordance with this embodiment, a is determined from a trial decoding of 
the original unmarked image I in a decoder 4. The decoding is that which would be 
used to decode the watermarked image I'. A value a' is produced by a calculator S3- 
S8. to which an offset value is added by an adder S9 to produce a. This produces 
values of a over the image, which are used to scale the data Ri so as to conceal the 
data. 

Figure 2 shows an example of the system of Figure 1 in which the calculation 
of a also encodes Ri. Figure 6 shows an example of the system of Figure 1 in which 
the trial decoding takes place after a trial processing of the image data by a process 
which is likely to be applied to the watermarked image. 

The data Ri in the examples given below is derived from a UMID. UMIDs are 
described in the section UMIDs below. As mentioned above, and in the examples 
given below, the image is subjected to a wavelet transform. Wavelet transforms are 
briefly discussed in the section Wavelets below. 

Figure 2 

Referring to Figure 2, for the purposes of explanation, it is assumed that the 
wavelet transform applied to the original spatial domain image results in a transform 
having four sub-bands of level 1: see the section Wavelets below. For ease of 
explanation the following description will refer only to the upper horizontal sub-band, 
but it will be appreciated that the present invention can be applied to any sub-band and 
may be applied to a plurality of sub-bands. The coefficients of the wavelet transform 
are denoted by Ci where i is the i th coefficient of a sequence of JxL coefficients where 
there are J bits of UMID data Wl to WJ. As will become apparent the JxL coefficients 
correspond to symbols Pi of a Pseudo Random Symbol Sequence PRSS. Each UMID 
bit is embedded in L wavelet coefficients. Watermark data bit Wj is embedded in 
coefficients Ci for i = (j-i)L+l to jL. 

In this example a UMID is produced as the watermark data by a UMID 
generator 6. In this example the watermark is imperceptible. The data of the UMID is 
combined with the wavelet coefficients Ci in an embedder 3 in the manner described 
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in detail in the following text. The transformed image together with the watermark Ci' 
is subjected to an inverse wavelet transform T\ (5) to produce a spatial domain 
watermarked image I'. 

The watermark is decoded and, optionally, removed from the watermarked 
image using the illustrative decoding and removal system of Figure 4. 

Trial decoder 4 

A trial decoder 4 comprises a generator 4], a converter 42 ; and a correlator SI . 
The generator 4i produces a pseudo random symbol sequence (PRSS) which is pseudo 
random because the sequence whilst appearing random can be reliably reproduced. 
The binary sequence has a length of JxL bits. The converter 4 2 converts the binary 1 
and 0 to +1 and -1 respectively to produce a pseudo random symbol sequence (PRSS) 
Pi of values +1 and -L The symbols of the PRSS are denoted herein by Pi, where i 
denotes the i th symbol of the sequence which is JxL symbols long. The correlator SI 
produces correlation values 

Sj = SCi.Pi, where the sum is taken over the range i = (j-l)L + 1 to jL for each 
of j=l to J. (See Figure 3 A.) 

Calculatio n of aj and embedding the UMID 

There is one strength value aj for each UMID bit Wj. 

The calculation of aj and the embedding of the UMID in the image operates in 
accordance with the flow diagram of Figure 3B. The embedder 3 calculates a function 
Ci' = Ci + aj Pi for i = (j-l)L + 1 to jL for each of j=l to J. 

where Ci' is an ith wavelet coefficient modified to encoded a bit of 
watermarking data; and 

aj is the scaling factor, the value of which depends on: 

a) the value 1 or 0 of a bit Wj of the UMID to be encoded in modified 
coefficient Ci'; and 

b) the sign of the correlation value Sj = ECi.Pi, for i = (j-l)L + 1 to jL for each 
of j=l to J, produced by the trial decoder 4: and 

c) the offset value, which is +/-1 in this example, so aj = aj' + offset). 
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The principle of operation is that a watermark bit Wj=l is encoded as a 
positive correlation value and Wj=0 is encoded as a negative correlation value ( or vice 
versa), aj is chosen to ensure the value of a correlation Sj' = ZCi'.Pi for i = (j-l)L + 1 
to jL for each of j=l to J, performed at the decoder has the correct sign to represent the 
5 value of bit Wj. If the correlation Sj performed at the encoder has the correct sign, 
then aj' = 0 otherwise aj" is modified to ensure that the correlation Sf = ZCi\Pi 
performed at the decoder has the correct sign. 

Thus referring to Figure 3B> 

A value aj"^aj- offsetj. 
10 Step SI (correlator SI of Figure 2) calculates the correlation value Sj = XCi.Pi, 

where the sum is taken over the range i = (j-l)L + 1 to jL for a sequence i = (j-l)L + 1 
to jL of coefficients Ci and PRSS bits Pi. (Note that 'symbols' Pi have values +1 and -1 
to ensure that bits of value 0 produced by the generator 4j contribute to the value of 
Sj.) Step SI is a trial decoding with a trial value of aj = 0. 
1 5 Step S2 determines whether the bit Wj of the UMID generated by generator 6 

is 1 or 0. It will be appreciated that the bit Wj is in effect encoded over L coefficients. 
If Wj =1 then steps S3 to S5 and S9"*" are followed. These steps are implemented by 
blocks S3-S8 and S9 in figure 2. 

Step S3 determines the sign of the correlation Sj. If the sign is positive and the 
20 bit Wj is 1 then 

at step S4 aj' =0. 

If the sign determined at step S3 is negative but the bit Wj =1 (which should be 
encoded by Sj positive), then 

at step S5aj'=-Sj/(L-1). 
25 At step S9 + . the offset +1 is added to ensure that aj is positive if SjK) and to 

increase robustness.. It should be noted that the offset is a signed value (+1) in this 
example. 

If Wj=0 then steps S6 to S8 and S9" are followed. These steps are implemented 
by blocks S3-S8 and S9 in figure 2. 
30 Step S6 determines the sign of the correlation Sj. If the sign is negative and the 

bit Wj is 0 then 
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at stepS7aj' =0. 

If the sign determined at step S6 is positive but the bit Wj =0 (which should be 
encoded by S negative), then 

at step S8 aj'=-Sj/(L-l). 
5 At step S9" the offset -1 is added to ensure that aj is negative if Sj=0 and to 

increase robustness. It should be noted that the offset is a signal value (-1) in this 
example. 

At step SI 0 the value Cf = Ci + aj Pi is calculated for i = (j-l)L + 1 to jL . 
The value aj"=-Sj/(L-l) is an example. The value aj' could be aj'=-Sj/L as 
1 0 another example. 

At step S9" an offset of + 1 could be subtracted from aj\ 

Watermark Decoding and Removing Svstem ( Figures 4 and 5) 
Referring to Figure 4, the watermark removing and decoding system has an 
input for receiving a spatial domain watermarked image I' from the system of Figure 1. 
15 The image may have been subject to image processing (not shown) between 
production by the system of Figure 1 and the receipt by the system of Figure 4. 

The received image is transformed by a wavelet transformer 46 (T) to produce 
wavelet coefficients Ci'. The coefficients Ci' are provided to a synchroniser 8 which 
correlates the coefficients with a PRSS generated by a generator 10. The synchroniser 
20 8 and the PRSS generator 10 carry out. in known manner, correlations with differing 
shifts of the PRSS relative to the coefficients to determine the position in the 
watermarked transformed image of the original PRSS produced at the watermarking 
system of Figure 1. Once synchronisation has been achieved the coefficients Cr are 
correlated with the PRSS in another correlator 12 to produce the correlation value 
25 Sj'=£Ci\Pi for i - (j-l)L + 1 to jL for each of j=l to J. 

where Pi has values +1 and -1. 

The correlation value Sj' is provided to a decoder 14 and to a remover 16, the 
operations of which will be described with reference to the flow diagram of Figure 5. 
30 The decoder 14 extracts the UMID from the image. The watermark is removed by the 
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remover 16. The resulting restored transformed image is subject to an inverse wavelet 
transform (T 1 ) in an inverse transformer 18. 

Referring to Figure 5, the synchronisation of the PRSS with the received 
transformed image occurs at step S12. At step S14. the correlation value 
5 Sy=£Ci\Pifori = (j-l)L+ 1 tojL. 



is calculated over a length L of the PRSS. 

At step SI 6. the sign of the value Sj ! is determined. If Sj ; is negative then the 
bit of the watermark, (the UMID in this example), is 0. If Sj* is positive the bit of the 
1 0 watermark is 1 . 

At step SI 8. 

otj=Sj'/(L-l) 

is calculated from Sj'. (This calculation may be an approximation because it 
O assumes that LCi.Pi =0) 

rll 

m 15 At step S22 ; Ci = Ci* - aj Pi is calculated for . i = (j-l)L + 1 tojL. 



If. in the embedding process, aj is calculated as - ~- at the step S5 or S8 ? then 



in the decoding process aj ? is calculated as Sj VL at step SI 8. 
Modifications. 

20 a). Threshold on the values of Ci and Ci 1 (Figure 1 1 A, Figure 12) 

In a modification of the embodiment described above, the values of the 
coefficients Ci are compared (S40) with a threshold value The at the embedder of 
Figure 2. and the values of the coefficients Ci' are compared S41 with a threshold 
value Thd at the remover 16 of Figure 4 and also at the decoder 14 of Figure 4. If the 

25 value of a coefficient exceeds the threshold, that coefficient is not used (S42. S43) in 
establishing the correlation value Sj or Sj'. The and Tha may be equal, but it has been 
found that Th d is preferably greater than The 

By way of a simple example, assume that the PRSS has length L=4 and 
symbols PI to P4 have values +1, -1,-1. and +L Then referring to Table 1 three 

30 examples are shown. 
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Table 1 



Example 1 (Exl) 

The coefficients Ci have values shown. If the value of the bit Wj of the 
watermark to be encoded is 0 then according to Figure 3, Sj=-1 and so aj y =0. 
Example 2 (Ex2) 

However if as shown in example 2 the coefficient C2 has a value -25 than 
Sj=+19 and aj '=-19/3. Large values of aj' may cause the watermark to be perceptible 
when it should be imperceptible. 

Example 3 (Ex3) 

In accordance with an embodiment of the present invention, thresholds + Th e 
and -The . are set. The magnitude of The may be about 6 for the above example. In 
practice it is set empirically. Thus as shown in Table 1. the coefficient C2 is not used 
in the calculation of Sj. and also the corresponding symbol of the PRBS is also not 
used. As a result Sj=-6 and aj'=0. Thus if the magnitude of a coefficient exceeds the 
threshold the coefficient is not used. 

Now, referring to Figure 1 1 A ? in accordance with this embodiment, the 
following procedure takes place at the embedder before step SI of Figure 3B. 

At step S40 ? the magnitude of the coefficient value Ci is compared with the 
threshold The If the magnitude of Ci is greater than the threshold The then at step S42 
Ci is not used. Otherwise at step S44 Ci is used to calculate Ci' as described with 
reference to Figure 3B. Referring to Figure 1 IB. it will be recalled that each symbol 
Pi of the PRSS is associated with a coefficient Ci. When a coefficient Ci is not used 
because it exceeds the threshold, the corresponding symbol Pi generated by the 
generator 4 is also not used as indicated by the blocks Ci and Pi in Figure 1 IB. 

b) Clip ping coefficient values (Figure 13) 



P10031US 14 

In an alternative modification, the values of the modified coefficients Ci' are 
clipped at the decoder of Figure 4 if they exceed (S80) a threshold value of magnitude 
Thchp. Thus coefficient values greater than the threshold are reduced to a 
predetermined value e.g. Th C h P . For example referring to Table 1 Example 2. the 
5 coefficient C2 (-25) is clipped to say +Th c i, p e.g. -6 at the decoder. If Ci>+T C hp 5 then C, 
is set to +T c iip (step S84). 

ITciip! ~ 6 is only an example and in practice may have other values set by 
experiment. 

Such clipping may or may not be performed also at the embedder of Figure 2. 
1 0 In the embedder shown in Figure 2 it is not performed. However, in another 

embodiment, the procedure of Figure 13 may be applied prior to step SI in Figure 3B. 

The clipping is performed only for the purpose of calculating the parameter a,. 
The coefficients C, to which a y P, is added do not have clipped values. 

Limiting or 

15 The value of a' may be limited to be within a present range determined by 

upper and lower bounds. 

Trial processing and decod ing- Figure 6 

Referring to Figure 6. an unmarked spatial domain image I is applied to an 
embedder 60. An example of the embedder is shown in Figure 8. The embedder 
20 calculates 

Ci" = Ci + octj Ri for i = (j-l)L + 1 to jL for each of j=l to J. 

where: Ci is a wavelet transform coefficient of the image; Ri is a 
watermarking symbol formed by combining a PRSS of JxL bits Pi with watermark 
25 data Wj. Symbol Ri has a value + or - 1 ; a tJ is a trial value of the scaling factor for 
UMID ( or watermark ) data bit Wj. In this example oty is initialised to 1 . Figure 7 
shows an example of a subsystem, for producing Ri. 

The embedder also includes an inverse transformer which produces a spatial 
domain watermarked image Iw. 
30 The image Iw is processed by a processor 62 to produce a processed spatial 

domain image Ip. The processor 62 is chosen to process the image according to a 
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process which the watermarked image is likely to encounter in use. and/or a process 
which may be applied to the image to deliberately remove or damage the watermark. 
JPEG processing using DCT transforms is a process which is known to be potentially 
damaging to some watermarks. 
5 A decoder 64. an example of which is shown in Figure 9. decodes the 

processed image Ip. The decoder may extract the watermark data Wj. The decoder 
produces correlation values 

Sip.j'= ZCip.Pi for i = (j-l)L + 1 to jL for each of j=l to J. 

1 0 where Cip are wavelet coefficients of the processed image Ip and the sum is 

calculated over a length L of a PRSS having JxL bits Pi. 

A calculator 66 calculates a new value of aj based on the magnitude of Sip j to 
produce new trial values ofa t +n :J which is used as a new value in the trial embedder 
60. n is the number of iterations used to calculate a final value a t+n j which is applied 
15 to an embedder 69. n = 0.1 .2. Several iterations may be used. Preferably the number 
of iterations is limited to a predetermined number, e.g. 4. because the process 62 may 
be non-linear ( JPEG processing is non-linear) and it is then unlikely that the iterations 
will converge to steady values of octj. 

Examples of (a) the calculator 66 and (b) the embedder 69 are shown in 
20 Figures 1 0 and 8 respectively. 

Calculating Ri Figure 7. 

A PRSS generator 71 produces a PRSS having JxL bits Pi. A UMID generator 
72 produces a UMID having bits Wj. In a modulator 73, each bit Wj of the UMID 
modulates, and is thus spread over, an L bit sequence of the PRSS. A data converter 
25 74 converts the binary values 1 and 0 at the output of the modulator to produce 
symbols Ri of value +1 and -1 respectively. 

Embedder 60 and 69.- Figure 8. 

The embedder of Figure 8 comprises a wavelet transformer 82 which produces 
the wavelet coefficients Ci and an inverse transformer 85. A multiplier 84 calculates 
30 a t +nj.Ri. An adder 83 adds a t + n j.Ri to Ci to produce. 

Ci' = Ci + a t +nj.Ri for i = (j-l)L + 1 to jL for each of j=l to J. 
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Thus each coefficient Ci is modified by a value of a associated with that 
coefficient and by one symbol Ri. 

Unlike the example of Figures 2 and 3, Ri is a symbol stream comprising the 
5 PRSS modulated by the data to be embedded, and a is an unsigned magnitude. 
Decoder 64-Figure 9 

The decoder has a wavelet transformer 91 which produces wavelet coefficients 
Cip from the processed image lp. A synchroniser 92 operating in known manner shifts 
the phase of the PRSS produced by a PRSS generator 93 so that it is in phase with the 
1 0 PRSS in the image lp. A data converter 94 converts the PRSS values Pi to +1 and -1. 
A correlator calculates a correlation value 

Sipj= ICip.Pi i - (j-l)L 4- 1 to jL for each of j=l to J. 

A decoder 96 determines the values of the data bits Wj from the sign of the 
1 5 correlation values Sipj. 

Calculating a-Figures 10A. 10B and IOC 

Referring to Figure 10A ; new values of a t+ ij are calculated by adding an offset 
to a basic fixed value ay in an adder 99. The offsets are produced by an offset 
generator 95. The generator responds to an offset control value produced by a 
20 processor 97. The processor controls the offset and thus the values of a t +ij in 
dependence on the correlation values Sip J. 

Figures 10B and 10C illustrate examples of the operation of the processor. 
Referring to Figure JOB, the correlation values Sipj are compared at step S30 
with the corresponding symbols Wj. The correlation values Sipj are positive and 
25 negative, a positive value indicates a symbol 1 and a negative value a symbol 0, (if the 
values Sip are unchanged by the processing in processor 62). If the signs of Sipj 
correctly represent Wj then the magnitude of Sipj is compared with an upper threshold 
Th. If | Sipj | >Th then the value of aj is reduced for the next iteration a m j. If 
I Sipj | is not greater than the threshold aj either remains unchanged for the next 
30 iteration ot t +i j or is used as the final value of cctj. 
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If the sign of Sip j indicates the incorrect value for Wj. then otj is increased for 
the next iteration <x t +ij. 

Referring to Figure IOC. at step S40 a value (Sipj)/L is calculated from Sipj. 
That is the average correlation value over L symbols. That value is used as a t+J j for 
5 the next iteration. Preferably a t +ij is compared with an upper threshold Th at step S42. 
If (Xt+ij exceeds Th, then a t+Jj is reduced. Otherwise it is compared (S46) with a lower 
threshold TL. If a t +i j is less than TL. a t +ij is increased (S48) otherwise it is 
unchanged (S49). 

Modifications. 
10 Other transforms 

Whilst the invention has been described by way of example with reference to 
Wavelet transforms, it may be used with other transforms for example DCT. 

Other materia] 

Whilst the invention has been described by way of example with reference to 
1 5 material comprising images, e.g. video material . it may be applied to other material, 
for example audio material and data material. 
Other Watermark data . 

Whilst the invention has been described by way of example with reference to 
UMIDs as the watermark data, it may be used with other data as the watermark. 
20 Wavelets 

Wavelets are well known and are described in for example "A Really Friendly 
Guide to Wavelets'* by C Valens, 1999 and available at 
http://perso.wanadoo.fr/polyvalens/clemens/wavelets/wavelets.html. 

Valens shows that the discrete wavelet transform can be implemented as an 
25 iterated filter bank as used in sub-band coding, with scaling of the image by a factor 
of 2 at each iteration. 

Thus referring to Figure 12, a spatial domain image is applied to a set of high 
pass HP and low pass LP filters. At level 1 ? the first stage of filtering, the image is 
filtered horizontally and vertically and, in each direction, scaled down by a factor of 2. 
30 In level 2, the low pass image from level 1 is filtered and scaled in the same way as in 
level 1. The filtering and scaling may be repeated in subsequent levels 3 onwards. 



P10031US 



18 



The result is shown schematically in Figure 1 1 . Figure 1 1 is a representation 
normal in the art. At level one the image is spatially filtered into four bands: the lower 
horizontal and vertical band, lHi, IV j: the upper horizontal band hHj, lVi; the upper 
vertical band lHj, hV 3 ; and the upper horizontal and vertical band. hHi, hVj. At level 
2. the lower horizontal and vertical band. 1H] ; IV i is filtered and scaled into the lower 
horizontal and vertical band. 1H 2 , 1V 2 : the upper horizontal band hH 2 , 1V 2 : the upper 
vertical band 1H 2 , hV 2 ; and the upper horizontal and vertical band, hH 2 . hV 2 At level 3 
(not shown in Figure 11), the lower horizontal and vertical band, 1H 2 , 1V 2 is farther 
filtered and scaled. 
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UMIDs 

The UMID or Unique Material Identifier is described in SMPTE Journal March 
2000. Referring to Figure 13. an extended UMID is shown. It comprises a first set of 
32 bytes of basic UMID and a second set of 32 bytes of signature metadata. 
5 The first set of 32 bytes is the basic UMID. The components are: 

•A 12-byte Universal Label to identify this as a SMPTE UMID. It defines the 
type of material which the UMID identifies and also defines the methods by which the 
globally unique Material and locally unique Instance numbers are created. 

•A 1 -byte length value to define the length of the remaining part of the UMID. 
10 # A 3-byte Instance number which is used to distinguish between different 

instances' of material with the same Material number. 

•A 16-byte Material number which is used to identify each clip. Each Material 
number is the same for related instances of the same material. 

The second set of 32 bytes of the signature metadata as a set of packed 
1 5 metadata items used to create an extended UMID. The extended UMID comprises the 
basic UMID followed immediately by signature metadata which comprises: 

•An 8-byte time/date code identifying the time and date of the Content Unit 
creation. 

•A 12-byte value which defines the spatial co-ordinates at the time of Content 
20 Unit creation. 

•3 groups of 4-byte codes which register the country, organisation and user 

codes 

Each component of the basic and extended UMIDs will now be defined in turn. 
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The 12-byte Universal Label 



The first 12 bytes of the UMID provide identification of the UMID by the 
registered string value defined in table 1 . 



Byte No. 


Description 


Value (hex) 


1 


Object Identifier 


06h 


2 


Label size 


OCh 


3 


Designation: ISO 


2Bb 


A 

4 


Designation: SMPTE 


34h 


5 


Registry: Dictionaries 


Olh 


6 


Registry: Metadata Dictionaries 


Olh 


7 


Standard: Dictionary Number 


01b 


8 


Version number 


Olh 


9 


Class: Identification and location 


Olh 


10 


Sub-class: Globally Unique Identifiers 


Olh 


11 


Type: UMID (Picture, Audio, Data, Group) 


01, 02, 03, 04h 


12 


Type: Number creation method 


XXh 



Table 1: Specification of the UMID Universal Label 
The hex values in table 1 may be changed: the values given are examples. 
Also the bytes 1-12 may have designations other than those shown by way of example 
in the table. Referring to the Table 1 , in the example shown byte 4 indicates that bytes 
5-12 relate to a data format agreed by SMPTE. Byte 5 indicates that bytes 6 to 10 
relate to "dictionary" data. Byte 6 indicates that such data is "metadata" defined by 
bytes 7 to 10. Byte 7 indicates the part of the dictionary containing metadata defined 
by bytes 9 and 10. Byte 10 indicates the version of the dictionary. Byte 9 indicates 
the class of data and Byte 10 indicates a particular item in the class. 

In the present embodiment bytes 1 to 1 0 have fixed preassigned values. Byte 
1 1 is variable. Thus referring to Figure 14, and to Table 1 above, it will be noted that 
the bytes 1 to 10 of the label of the UMID are fixed. Therefore they may be replaced 
by a 1 byte 'Type" code T representing the bytes 1 to 10. The type code T is followed 
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by a length code L. That is followed by 2 bytes, one of which is byte 11 of Table 1 
and the other of which is byte 12 of Table 1, an instance number (3 bytes) and a 
material number (16 bytes). Optionally, the material number may be followed by the 
signature metadata of the extended UMID and/or other metadata. 
5 The UMID type (byte 1 1) has 4 separate values to identify each of 4 different 

data types as follows: 

'Ollv = UMID for Picture material 

4 02h* - UMID for Audio material 

4 03h" = UMID for Data materia] 
1 0 6 04h' = UMID for Group material (i.e. a combination of related essence). 

The last (12th) byte of the 12 byte label identifies the methods by which the 
material and instance numbers are created. This byte is divided into top and bottom 
nibbles where the top nibble defines the method of Material number creation and the 
bottom nibble defines the method of Instance number creation. 
1 5 Length 

The Length is a 1-byte number with the value '13h' for basic UMIDs and c 33h ; 
for extended UMIDs. 

Instance Number 

The Instance number is a unique 3-byte number which is created by one of 
20 several means defined by the standard. It provides the link between a particular 
'instance* of a clip and externally associated metadata. Without this instance number, 
all material could be linked to any instance of the material and its associated metadata. 

The creation of a new clip requires the creation of a new Material number 
together with a zero Instance number. Therefore, a non-zero Instance number 
25 indicates that the associated clip is not the source material. An Instance number is 
primarily used to identify associated metadata related to any particular instance of a 
clip. 

Material Number 

The 16-byte Material number is a non-zero number created by one of several 
30 means identified in the standard. The number is dependent on a 6-byte registered port 
ID number, time and a random number generator. 
Signature Metadata 
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Any component from the signature metadata may be null-filled where no 
meaningful value can be entered. Any null-filled component is wholly null-filled to 
clearly indicate a downstream decoder that the component is not valid. 

The Time-Date Format 
5 The date-time format is 8 bytes where the first 4 bytes are a UTC (Universal 

Time Code) based time component. The time is defined either by an AES3 32-bit 
audio sample clock or SMPTE 12M depending on the essence type. 

The second 4 bytes define the date based on the Modified Julian Data (MJD) as 
defined in SMPTE 309M. This counts up to 999,999 days after midnight on the 17th 
1 0 November 1 858 and allows dates to the year 4597. 

The Spatial Co-ordinate Format 

The spatial co-ordinate value consists of three components defined as follows: 

•Altitude: 8 decimal numbers specifying up to 99,999,999 metres. 

•Longitude: 8 decimal numbers specifying East/West 180.00000 degrees (5 
1 5 decimal places active). 

•Latitude: 8 decimal numbers specifying North/South 90.00000 degrees (5 
decimal places active). 

The Altitude value is expressed as a value in metres from the centre of the earth 
thus allowing altitudes below the sea level. 
20 It should be noted that although spatial co-ordinates are static for most clips, 

this is not true for all cases. Material captured from a moving source such as a camera 
mounted on a vehicle may show changing spatial co-ordinate values. 

Country Code 

The Country code is an abbreviated 4-byte alpha-numeric string according to 
25 the set defined in ISO 3 1 66. Countries which are not registered can obtain a registered 
alpha-numeric string from the SMPTE Registration Authority. 
Organisation Code 

The Organisation code is an abbreviated 4-byte alpha-numeric string registered 
with SMPTE. Organisation codes have meaning only in relation to their registered 
30 Country code so that Organisation codes can have the same value in different 
countries. 

User Code 
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The User code is a 4-byte alpha-numeric string assigned locally by each 
organisation and is not globally registered. User codes are defined in relation to their 
registered Organisation and Country codes so that User codes may have the same 
value in different organisations and countries. 

Although illustrative embodiments of the invention have been described in detail 
herein with reference to the accompanying drawings, it is to be understood that the 
invention is not limited to those precise embodiments, and that various changes and 
modifications can be effected therein by one skilled in the art without departing from the 
scope and spirit of the invention as defined by the appended claims. 



